Redis 3.2.0-Sentinel故障转移测试

发表于   |   分类于   |     |   阅读次数

废话少说，直接给配置，注意本文的ip是虚构的.

一. 拓扑

一主一从，三个sentinel

角色 地址
master 10.10.xx.1:6379
slave 10.10.xx.2:6379
sentinel-1 10.10.xx.3:26379
sentinel-2 10.10.xx.4:26379
sentinel-3 10.10.xx.5:26379

二. 配置

1
本文为了测试，只给出部分重点配置。

注意：Redis 3.2.0默认bind是127.0.0.1，并且开启保护模式，如果搭建sentinel、cluster应该进行修改。
有关保护模式：可以参考：http://carlosfu.iteye.com/blog/2303254

1. master(10.10.xx.1:6379)

1
2
3
4
5
6
7
8
9
10
port 6379
daemonize yes
pidfile "/var/run/redis-6379.pid"
logfile "6379.log"
dbfilename "dump-6379.rdb"
appendonly yes
appendfilename "appendonly-6379.aof"
dir "/opt/soft/redis-3.2.0/data"
bind 0.0.0.0
protected-mode no

2. slave(10.10.xx.2:6379)

1
2
3
4
5
6
7
8
9
10
11
port 6379
daemonize yes
pidfile "/var/run/redis-6379.pid"
logfile "6379.log"
dbfilename "dump-6379.rdb"
appendonly yes
appendfilename "appendonly-6379.aof"
dir "/opt/soft/redis-3.2.0/data"
bind 0.0.0.0
protected-mode no
slaveof 10.10.xx.1 6379
1
三个sentinel的配置是一样的

3. sentinel-1(10.10.xx.3:26379)

1
2
3
4
5
6
7
8
9
10
11
port 26379
daemonize yes  
pidfile /var/run/redis-26379.pid  
logfile "26379.log"  
dir /opt/soft/redis-3.2.0/data/    
sentinel monitor mymaster 10.10.xx.1 6379 2   
sentinel down-after-milliseconds mymaster 30000  
sentinel parallel-syncs mymaster 1  
sentinel failover-timeout mymaster 180000 
bind 0.0.0.0
protected-mode no

4. sentinel-2(10.10.xx.4:26379)

1
2
3
4
5
6
7
8
9
10
11
port 26379
daemonize yes  
pidfile /var/run/redis-26379.pid  
logfile "26379.log"  
dir /opt/soft/redis-3.2.0/data/    
sentinel monitor mymaster 10.10.xx.1 6379 2   
sentinel down-after-milliseconds mymaster 30000  
sentinel parallel-syncs mymaster 1  
sentinel failover-timeout mymaster 180000 
bind 0.0.0.0
protected-mode no

5. sentinel-3(10.10.xx.5:26379)

1
2
3
4
5
6
7
8
9
10
11
port 26379
daemonize yes  
pidfile /var/run/redis-26379.pid  
logfile "26379.log"  
dir /opt/soft/redis-3.2.0/data/    
sentinel monitor mymaster 10.10.xx.1 6379 2   
sentinel down-after-milliseconds mymaster 30000  
sentinel parallel-syncs mymaster 1  
sentinel failover-timeout mymaster 180000 
bind 0.0.0.0
protected-mode no

三、启动

1.启动master
2.启动slave
3.启动三个sentinel

四、模拟故障

1.在10.10.xx.1上，直接kill -9 ${master的pid}，观察一会，发现10.10.xx.2:6379变为master
2.重新启动源master(10.10.xx.1:6379),观察，它变为了10.10.xx.2:6379的slave。

具体过程看一下日志，比较清楚

五、完整的日志

1. master

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 30853
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               
30853:M 14 Jun 16:56:34.335 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
30853:M 14 Jun 16:56:34.335 # Server started, Redis version 3.2.0
30853:M 14 Jun 16:56:34.335 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
30853:M 14 Jun 16:56:34.335 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
30853:M 14 Jun 16:56:34.335 * The server is now ready to accept connections on port 6379
30853:M 14 Jun 16:59:14.397 * Slave 10.10.xx.2:6379 asks for synchronization
30853:M 14 Jun 16:59:14.397 * Full resync requested by slave 10.10.xx.2:6379
30853:M 14 Jun 16:59:14.397 * Starting BGSAVE for SYNC with target: disk
30853:M 14 Jun 16:59:14.398 * Background saving started by pid 31065
31065:C 14 Jun 16:59:14.399 * DB saved on disk
31065:C 14 Jun 16:59:14.400 * RDB: 6 MB of memory used by copy-on-write
30853:M 14 Jun 16:59:14.412 * Background saving terminated with success
30853:M 14 Jun 16:59:14.413 * Synchronization with slave 10.10.xx.2:6379 succeeded
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 32387
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               
32387:M 14 Jun 17:12:40.086 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
32387:M 14 Jun 17:12:40.087 # Server started, Redis version 3.2.0
32387:M 14 Jun 17:12:40.087 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
32387:M 14 Jun 17:12:40.087 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
32387:M 14 Jun 17:12:40.087 * DB loaded from append only file: 0.000 seconds
32387:M 14 Jun 17:12:40.087 * The server is now ready to accept connections on port 6379
32387:S 14 Jun 17:12:50.337 * SLAVE OF 10.10.xx.2:6379 enabled (user request from 'id=2 addr=10.10.xx.3:33844 fd=6 name=sentinel-090610f6-cmd age=10 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=0 qbuf-free=32768 obl=36 oll=0 omem=0 events=r cmd=exec')
32387:S 14 Jun 17:12:50.337 # CONFIG REWRITE executed with success.
32387:S 14 Jun 17:12:51.113 * Connecting to MASTER 10.10.xx.2:6379
32387:S 14 Jun 17:12:51.114 * MASTER <-> SLAVE sync started
32387:S 14 Jun 17:12:51.114 * Non blocking connect for SYNC fired the event.
32387:S 14 Jun 17:12:51.114 * Master replied to PING, replication can continue...
32387:S 14 Jun 17:12:51.115 * Partial resynchronization not possible (no cached master)
32387:S 14 Jun 17:12:51.116 * Full resync from master: a4aeea7ef3dadfed876e6f6479a5394ba8e113e6:1
32387:S 14 Jun 17:12:51.158 * MASTER <-> SLAVE sync: receiving 94 bytes from master
32387:S 14 Jun 17:12:51.158 * MASTER <-> SLAVE sync: Flushing old data
32387:S 14 Jun 17:12:51.158 * MASTER <-> SLAVE sync: Loading DB in memory
32387:S 14 Jun 17:12:51.158 * MASTER <-> SLAVE sync: Finished with success
32387:S 14 Jun 17:12:51.160 * Background append only file rewriting started by pid 32390
32387:S 14 Jun 17:12:51.184 * AOF rewrite child asks to stop sending diffs.
32390:C 14 Jun 17:12:51.185 * Parent agreed to stop sending diffs. Finalizing AOF...
32390:C 14 Jun 17:12:51.185 * Concatenating 0.00 MB of AOF diff received from parent.
32390:C 14 Jun 17:12:51.185 * SYNC append only file rewrite performed
32390:C 14 Jun 17:12:51.186 * AOF rewrite: 6 MB of memory used by copy-on-write
32387:S 14 Jun 17:12:51.214 * Background AOF rewrite terminated with success
32387:S 14 Jun 17:12:51.214 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
32387:S 14 Jun 17:12:51.214 * Background AOF rewrite finished successfully

2. slave

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 13854
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               
13854:S 14 Jun 16:59:14.377 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
13854:S 14 Jun 16:59:14.377 # Server started, Redis version 3.2.0
13854:S 14 Jun 16:59:14.377 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
13854:S 14 Jun 16:59:14.377 * The server is now ready to accept connections on port 6379
13854:S 14 Jun 16:59:14.377 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 16:59:14.377 * MASTER <-> SLAVE sync started
13854:S 14 Jun 16:59:14.378 * Non blocking connect for SYNC fired the event.
13854:S 14 Jun 16:59:14.378 * Master replied to PING, replication can continue...
13854:S 14 Jun 16:59:14.378 * Partial resynchronization not possible (no cached master)
13854:S 14 Jun 16:59:14.381 * Full resync from master: 6aeb689f547f20db04b3fa7ad2fa36d56dd32098:1
13854:S 14 Jun 16:59:14.395 * MASTER <-> SLAVE sync: receiving 76 bytes from master
13854:S 14 Jun 16:59:14.395 * MASTER <-> SLAVE sync: Flushing old data
13854:S 14 Jun 16:59:14.395 * MASTER <-> SLAVE sync: Loading DB in memory
13854:S 14 Jun 16:59:14.395 * MASTER <-> SLAVE sync: Finished with success
13854:S 14 Jun 16:59:14.396 * Background append only file rewriting started by pid 13857
13854:S 14 Jun 16:59:14.419 * AOF rewrite child asks to stop sending diffs.
13857:C 14 Jun 16:59:14.419 * Parent agreed to stop sending diffs. Finalizing AOF...
13857:C 14 Jun 16:59:14.419 * Concatenating 0.00 MB of AOF diff received from parent.
13857:C 14 Jun 16:59:14.419 * SYNC append only file rewrite performed
13857:C 14 Jun 16:59:14.420 * AOF rewrite: 6 MB of memory used by copy-on-write
13854:S 14 Jun 16:59:14.477 * Background AOF rewrite terminated with success
13854:S 14 Jun 16:59:14.477 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
13854:S 14 Jun 16:59:14.477 * Background AOF rewrite finished successfully
13854:S 14 Jun 17:11:34.635 # Connection with master lost.
13854:S 14 Jun 17:11:34.635 * Caching the disconnected master state.
13854:S 14 Jun 17:11:34.686 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:34.686 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:34.686 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:35.687 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:35.688 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:35.688 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:36.690 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:36.690 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:36.690 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:37.692 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:37.692 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:37.692 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:38.693 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:38.693 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:38.693 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:39.695 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:39.696 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:39.696 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:40.698 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:40.699 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:40.699 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:41.700 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:41.700 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:41.700 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:42.702 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:42.702 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:42.702 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:43.703 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:43.703 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:43.703 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:44.705 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:44.705 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:44.705 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:45.707 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:45.707 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:45.708 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:46.708 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:46.708 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:46.708 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:47.711 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:47.711 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:47.712 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:48.713 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:48.713 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:48.713 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:49.716 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:49.716 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:49.716 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:50.717 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:50.717 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:50.717 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:51.718 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:51.718 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:51.719 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:52.720 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:52.720 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:52.720 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:53.722 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:53.722 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:53.722 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:54.724 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:54.724 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:54.725 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:55.726 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:55.726 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:55.726 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:56.729 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:56.729 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:56.729 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:57.731 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:57.731 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:57.731 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:58.734 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:58.735 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:58.735 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:11:59.738 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:11:59.738 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:11:59.738 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:12:00.739 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:12:00.740 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:12:00.740 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:12:01.740 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:12:01.740 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:12:01.740 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:12:02.741 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:12:02.741 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:12:02.741 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:12:03.742 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:12:03.742 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:12:03.743 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:12:04.744 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:12:04.744 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:12:04.744 # Error condition on socket for SYNC: Connection refused
13854:S 14 Jun 17:12:05.746 * Connecting to MASTER 10.10.xx.1:6379
13854:S 14 Jun 17:12:05.746 * MASTER <-> SLAVE sync started
13854:S 14 Jun 17:12:05.746 # Error condition on socket for SYNC: Connection refused
13854:M 14 Jun 17:12:05.883 * Discarding previously cached master state.
13854:M 14 Jun 17:12:05.883 * MASTER MODE enabled (user request from 'id=6 addr=10.10.xx.4:56314 fd=9 name=sentinel-92aed965-cmd age=257 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=14 qbuf-free=32754 obl=36 oll=0 omem=0 events=r cmd=exec')
13854:M 14 Jun 17:12:05.883 # CONFIG REWRITE executed with success.
13854:M 14 Jun 17:12:51.082 * Slave 10.10.xx.1:6379 asks for synchronization
13854:M 14 Jun 17:12:51.082 * Full resync requested by slave 10.10.xx.1:6379
13854:M 14 Jun 17:12:51.082 * Starting BGSAVE for SYNC with target: disk
13854:M 14 Jun 17:12:51.083 * Background saving started by pid 15544
15544:C 14 Jun 17:12:51.084 * DB saved on disk
15544:C 14 Jun 17:12:51.084 * RDB: 6 MB of memory used by copy-on-write
13854:M 14 Jun 17:12:51.125 * Background saving terminated with success
13854:M 14 Jun 17:12:51.125 * Synchronization with slave 10.10.xx.1:6379 succeeded

3. sentinel-1

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 12259
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               
12259:X 14 Jun 17:05:04.278 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
12259:X 14 Jun 17:05:04.281 # Sentinel ID is 090610f6b1d240d4e9ae3b83e12a35225e513214
12259:X 14 Jun 17:05:04.282 # +monitor master mymaster 10.10.xx.1 6379 quorum 2
12259:X 14 Jun 17:05:04.283 * +slave slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
12259:X 14 Jun 17:07:50.891 * +sentinel sentinel 92aed96581eb5356b01901870d6cb5d8fcacdb29 10.10.xx.4 26379 @ mymaster 10.10.xx.1 6379
12259:X 14 Jun 17:09:44.208 * +sentinel sentinel 349f4052eae26b0f81e0edd823df5b8d095ec239 10.10.xx.5 26379 @ mymaster 10.10.xx.1 6379
12259:X 14 Jun 17:12:04.665 # +sdown master mymaster 10.10.xx.1 6379
12259:X 14 Jun 17:12:04.765 # +new-epoch 1
12259:X 14 Jun 17:12:04.768 # +vote-for-leader 92aed96581eb5356b01901870d6cb5d8fcacdb29 1
12259:X 14 Jun 17:12:05.774 # +odown master mymaster 10.10.xx.1 6379 #quorum 3/2
12259:X 14 Jun 17:12:05.774 # Next failover delay: I will not start a failover before Tue Jun 14 17:18:05 2016
12259:X 14 Jun 17:12:07.930 # -odown master mymaster 10.10.xx.1 6379
12259:X 14 Jun 17:12:07.939 # +config-update-from sentinel 349f4052eae26b0f81e0edd823df5b8d095ec239 10.10.xx.5 26379 @ mymaster 10.10.xx.1 6379
12259:X 14 Jun 17:12:07.939 # +switch-master mymaster 10.10.xx.1 6379 10.10.xx.2 6379
12259:X 14 Jun 17:12:07.940 * +slave slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
12259:X 14 Jun 17:12:37.971 # +sdown slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
12259:X 14 Jun 17:12:40.297 # -sdown slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
12259:X 14 Jun 17:12:50.293 * +convert-to-slave slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379

4. sentinel-2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 29115
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               
29115:X 14 Jun 17:07:48.865 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
29115:X 14 Jun 17:07:48.876 # Sentinel ID is 92aed96581eb5356b01901870d6cb5d8fcacdb29
29115:X 14 Jun 17:07:48.876 # +monitor master mymaster 10.10.xx.1 6379 quorum 2
29115:X 14 Jun 17:07:48.878 * +slave slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:07:49.188 * +sentinel sentinel 090610f6b1d240d4e9ae3b83e12a35225e513214 10.10.xx.3 26379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:09:44.204 * +sentinel sentinel 349f4052eae26b0f81e0edd823df5b8d095ec239 10.10.xx.5 26379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:04.685 # +sdown master mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:04.756 # +odown master mymaster 10.10.xx.1 6379 #quorum 2/2
29115:X 14 Jun 17:12:04.756 # +new-epoch 1
29115:X 14 Jun 17:12:04.756 # +try-failover master mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:04.759 # +vote-for-leader 92aed96581eb5356b01901870d6cb5d8fcacdb29 1
29115:X 14 Jun 17:12:04.763 # 349f4052eae26b0f81e0edd823df5b8d095ec239 voted for 92aed96581eb5356b01901870d6cb5d8fcacdb29 1
29115:X 14 Jun 17:12:04.765 # 090610f6b1d240d4e9ae3b83e12a35225e513214 voted for 92aed96581eb5356b01901870d6cb5d8fcacdb29 1
29115:X 14 Jun 17:12:04.826 # +elected-leader master mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:04.826 # +failover-state-select-slave master mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:04.897 # +selected-slave slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:04.897 * +failover-state-send-slaveof-noone slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:05.870 * +failover-state-wait-promotion slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:06.828 # +promoted-slave slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:06.829 # +failover-state-reconf-slaves master mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:06.877 # +failover-end master mymaster 10.10.xx.1 6379
29115:X 14 Jun 17:12:06.877 # +switch-master mymaster 10.10.xx.1 6379 10.10.xx.2 6379
29115:X 14 Jun 17:12:06.878 * +slave slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
29115:X 14 Jun 17:12:36.948 # +sdown slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
29115:X 14 Jun 17:12:41.114 # -sdown slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379

5. sentinel-3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 3.2.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 8413
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               
8413:X 14 Jun 17:09:42.233 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
8413:X 14 Jun 17:09:42.241 # Sentinel ID is 349f4052eae26b0f81e0edd823df5b8d095ec239
8413:X 14 Jun 17:09:42.241 # +monitor master mymaster 10.10.xx.1 6379 quorum 2
8413:X 14 Jun 17:09:42.244 * +slave slave 10.10.xx.2:6379 10.10.xx.2 6379 @ mymaster 10.10.xx.1 6379
8413:X 14 Jun 17:09:43.456 * +sentinel sentinel 090610f6b1d240d4e9ae3b83e12a35225e513214 10.10.xx.3 26379 @ mymaster 10.10.xx.1 6379
8413:X 14 Jun 17:09:43.867 * +sentinel sentinel 92aed96581eb5356b01901870d6cb5d8fcacdb29 10.10.xx.4 26379 @ mymaster 10.10.xx.1 6379
8413:X 14 Jun 17:12:04.826 # +sdown master mymaster 10.10.xx.1 6379
8413:X 14 Jun 17:12:04.835 # +new-epoch 1
8413:X 14 Jun 17:12:04.836 # +vote-for-leader 92aed96581eb5356b01901870d6cb5d8fcacdb29 1
8413:X 14 Jun 17:12:04.909 # +odown master mymaster 10.10.xx.1 6379 #quorum 3/2
8413:X 14 Jun 17:12:04.909 # Next failover delay: I will not start a failover before Tue Jun 14 17:18:04 2016
8413:X 14 Jun 17:12:07.025 # +config-update-from sentinel 92aed96581eb5356b01901870d6cb5d8fcacdb29 10.10.xx.4 26379 @ mymaster 10.10.xx.1 6379
8413:X 14 Jun 17:12:07.026 # +switch-master mymaster 10.10.xx.1 6379 10.10.xx.2 6379
8413:X 14 Jun 17:12:07.026 * +slave slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
8413:X 14 Jun 17:12:37.032 # +sdown slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379
8413:X 14 Jun 17:12:41.054 # -sdown slave 10.10.xx.1:6379 10.10.xx.1 6379 @ mymaster 10.10.xx.2 6379